Mitigating Asymmetric Nonlinear Weight Update Effects in Hardware Neural Network based on Analog Resistive Synapse

摘要

Asymmetric nonlinear weight update is considered as one of the majorobstacles for realizing hardware neural networks based on analog resistivesynapses because it significantly compromises the online training capability.This paper provides new solutions to this critical issue throughco-optimization with the hardware-applicable deep-learning algorithms. Newinsights on engineering activation functions and a threshold weight updatescheme effectively suppress the undesirable training noise induced byinaccurate weight update. We successfully trained a two-layer perceptronnetwork online and improved the classification accuracy of MNIST handwrittendigit dataset to 87.8/94.8% by using 6-bit/8-bit analog synapses, respectively,with extremely high asymmetric nonlinearity.